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CHAPTER  I 
STATEMENT  AND  JUSTIFICATION 
OF  THE  PROBLEM 


CHAPTER  I 

STATEMENT  AND  JUSTIFICATION  OF  THE  PROBLEM 


STATEMENT  OF  THE  PROBLEM.-  -The  problem  is  to  evaluate  the  Ele- 
mentary Evaluative  Criteria  which  was  copyrighted  in  1953  by  Dr.  James 
F.  Baker  of  Boston  University's  School  of  Education.    Specifically  this 
evaluation  is  to  determine  how  the  Elementary  Evaluative  Criteria  has 
aided  superintendents ,  supervisors,  administrators,  teachers,  and  lay 
personnel  in  evaluating  their  total  elementary  school  program  or  any  of 
its  parts. 

JUSTIFICATION  OF  THE  PROBLEM. -  -As  a  result  of  six  years  of 
experiment,  analysis,  and  refinement  of  materials  and  procedures,  the 
Cooperative  Study  of  Secondary-School  Standards  developed  an  Evaluative 
Criteria  for  the  secondary  school  in  19U0.    In  195>0  a  revised  form  of 
Evaluative  Criteria  was  authorized  for  publication. 

The  Evaluative  Criteria  have  been  used  very  extensively  by  second- 
ary schools  and  have  proved  of  great  value  in  effecting  improved  proce- 
dures and  methods  in  the  secondary  school  program. 

A  seminar  group  at  Boston  University  in  195>2  undertook  the  task 
of  developing  criteria  for  various  areas  of  the  elementary  school. 

Each  member  of  the  group  was  given  a  specific  area  in  the  elemen 
tary  school  to  evaluate  and  analyze.    Each  member's  purpose  was  to 
develop  a  set  of  criteria  to  evaluate  his  particular  area.    The  resulting 


work  of  each  member,  prior  to  the  termination  of  his  study,  was  sent  to 
prominent  specialists  in  that  field  or  area  for  evaluation  and  criticism. 

In  19^3  Dr.  James  F.  Baker  constructed  an  Elementary  Evaluative 
Criteria  comprised  of  the  criteria  developed  by  the  seminar  group  men- 
tioned above.    He  analyzed,  revised,  and  compiled  these  separate  criteria 
into  one  criteria  which  he  copyrighted  and  published  under  the  name 

Elementary  Evaluative  Criteria. 
— — — — 

In  order  for  any  evaluative  instrument  to  be  effective  it  should 
be  revised  from  time  to  time  in  light  of  changing  concepts.  "Any 
evaluation  program  should  be  subject  to  revision.    It  must  be  responsive,1 

i/ 

says  Ostrander    ,  "to  changes  in  the  curriculum  and  to  conditions  outside 
the  school  which  affect  children.    It  should  be  flexible  so  that  it  can 
be  modified  in  accordance  with  advances  in  the  techniques  of  evaluation." 
195>U  was  chosen  as  the  time  to  evaluate  the  Elementary  Evaluative  Crite- 
ria as  to  its  effectiveness  a.s  a  measuring  instrument  of  the  elementary 
school. 

This  study  has  been  undertaken  to  determine  if  the  Elementary 
Evaluative  Criteria  has  fulfilled  its  mission  of  helping  school  personnel 
evaluate  the  school  program. 


1/  R.  H.  Ostrander,  "Evaluation  in  the  Oak  Ridge  Schools,"  Educational 
Leadership,  (November,  19^0 )  8:86-90.  p.  88. 


CHAPTER  II 
REVIEW  OF  RESEARCH 


CHAPTER  II 
REVIEW  OF  RESEARCH 


RESEARCH  PERTAINING  TO  EVALUATION.-  -"The  nature  of  recent 

evaluation  techniques  is  very  intimately  related  to  the  concept  of  the 

1/  2/ 

role  and  purpose  of  evaluation  underlying  them. "      Taba    lists  some  of 

the  basic  characteristics  of  the  current  evaluation  practices: 

1)  Modern  curriculum  is  concerned  with  the  growth  of  the 
whole  child,  no  matter  how  inarticulately  and  unclearly  this  pur- 
pose may  be  perceived  and  no  matter  how  incompletely  the  practice 
may  live  up  to  it.    2)  Consequently,  in  evaluation,  serious  effort 
is  being  made  to  appraise  as  wide  a  range  of  educational  outcomes 
as  is  considered  to  be  significant  objectives  of  student  growth. 
3)  Thus,  besides  the  mastery  of  information  and  basic  skills, 
attention  is  devoted  to  ways  of  getting  adequate  and  objective 
evidence  of  development  of  interests,  desirable  social  attitudes, 
critical  thinking,  social  adjustment,  and  increasingly  intelligent 
tools  and  techniques  of  learning  and  study  .  .  •  k)  Newer  evalu- 
ation practices  also  attempt  to  appraise  desirable  changes  in 
behavior  processes  in  place  of  the  mastery  of  content.    5)  It  is 
equally  important  to  so  treat  the  evaluation  data  that  each  bit 
of  it  is  interpretated  in  the  light  of  all  other  available  evi- 
dence. 

3/ 

Wrightstone    says,  "Modern  evaluation  is  characterized  by  several 
criteria.    First,  it  attempts  to  measure  a  comprehensive  range  of 
objectives  of  the  modern  school  curriculum  rather  than  limited  subject 
matter  achievement  only. " 


1/  Hilda  Taba,  "Current  Evaluation  Techniques, 11  Childhood  Education, 
(September,  19Ul)  l8:lh-20.  p.LU. 

2/  Ibid.,  pp.  lli-l£. 

3/  J.  W.  Wrightstone,  "Trends  in  Evaluation,"  Educational  Leadership, 
(November,  19^0)  8:91-95.  p. 91. 


1/ 

Hammond  and  Drummond    believe  that: 

In  looking  for  the  good  elementary  school,  we  shall  probably 
be  in  search  of  something  that  we  shall  never  quite  find — at  least 
in  a  literal  or  absolute  sense.     Good  is  after  all  only  a  relative 
term,  and  schools — like  individuals,  will  probably  have  many  strong 
points,  and  at  least  a  few  weak  points.    No  matter  how  good  a  school 
may  be  today,  it  has  a  chance  of  being  better  or  worse  tomorrow. 
Many  factors  go  into  the  making  of  a  good  school,  some  of  which 
are  beyond  the  power  of  school  people  to  provide.    Schools,  like 
individual  pupils,  should  be  evaluated,  always  in  terms  of  their 
purposes  and  how  well  they  are  discharging  obligations  in  terms 
of  what  they  have  to  work  with. 

"In  general,  the  techniques  of  evaluation  and  measurement,"  says 
2/ 

Remmers  ,  "are  based  upon  the  very  useful  theory  and  applications  of 
sampling  and  inferential  statistics." 

"Although  a  great  deal  has  been  written  about  evaluation,"  says 

3/ 

Shane  ,  "and  the  evaluative  process  in  the  past  ten  or  fifteen  years 
the  question  of  what  actually  constitutes  appraisal  in  modern  education 


is  not  yet  a  matter  of  common  agreement  in  educational  practice." 

Shane  and  Rovner    conducted  a  survey  m  1950  to  determine  seme 

of  the  interpretations  of  evaluation  and  they  concluded  that: 

The  survey  suggested  that  five  meanings  have  attached  them- 
selves to  the  term:  (1)  evaluation  as  a  synonym  for  a  testing 
program,  (2)  evaluation  as  a  means  of  guaging  competence  of 


1/  Sarah  L.  Hammond  and  Harold  D.  Drummond,  "A  Region  Evaluates  its 
Elementary  Schools,"  Educational  Leadership,  (November,  195>0)  8:95-100. 
Quoted  from  the  Tentative  edition  of  Elementary  Evaluative  Criteria. 
Vol.  I,  p. 21. 

2/  H.  H.  Remmers,  "Evaluation  in  Curriculum  Development,"  School  and 
Society,  (July,  19k9)  70:33-37.  p. 35. 

3/  Harold  G.  Shane,  "A  195>0  Census  of  Evaluation  Practices,"  Educational 
Leadership,  (November,  1950)  8:73-77.  p. 73. 


k/  Ibid.,  p. 73. 


individual  teachers,  (3)  evaluation  as  a  procedure  followed  in 
judging  effectiveness  of  the  over-all  program  of  a  school,  (1+) 
evaluation  as  an  expression  of  values  in  the  form  of  criteria  to 
be  used  subsequently  in  studying  specific  problems,  practices  or 
conditions  in  the  classroom,  administration  or  curriculum,  and 
(5)  evaluation  as  a  process  by  means  of  which  changes  in  behavior 
of  children  are  studied  and  guided  toward  goals  sought  by  a  school. 

A  Florida  State  University  elementary  workshop  group,  during  the 
summer  of  19h9s  explored  the  field  of  evaluation,  to  provide  for  self- 
appraisal,  and  to  suggest  means  for  evaluating  phases  of  the  program 

y 

usually  considered  intangible,    Hammond  and  Drummond    tell  us  that  this 

group  developed  several  ideas: 

As  the  workshop  progressed,  participants  agreed  that  evalu- 
ation involves  at  least  three  steps:  (1)  stating  values  or  purposes 
which  the  school  deems  important,  (2)  securing  evidences  regarding 
the  extent  to  which  the  values  or  purposes  are  realized  in  practice, 
and  (3)  planning  ways  in  which  the  values  or  purposes  might  be 
better  achieved  or  more  fully  realized  and  related.    The  develop- 
ment of  an  awareness  of  values,  therefore,  was  seen  to  be  the  heart 
of  learning. 

A  review  of  the  data  listed  previously  seems  to  indicate  that 
one  may  crystallize  the  principles  of  evaluation  in  the  following 
manner.    Before  an  evaluation  is  attempted  there  should  be  agreed  upon 
values  and  objectives  to  guide  the  evaluator.    Single  words  or  works 
should  be  eliminated  and  in  their  places  words  in  "diagnostic ally  mean- 
ingful terms"  should  be  substituted.    It  should  be  kept  in  mind  that 
evaluation  is  a  continuous  process  that  is  a  direct  part  of  the  teaching- 
learning  situation  and  should  cover  a  long  period  of  time  rather  than 
a  short  period. 


1/  Sarah  L.  Hammond  and  Harold  D.  Drummond,  op.  cit. ,  pp.  96-97. 

2/  H.  J.  McNally,  "Evaluation— Of  Vfhat?  For  Miat?",  Educational  Admin- 
istration and  Supervision,  (January,  19h9)  35:36-U8 
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In  evaluating  elementary  schools,  Henry    cites  the  findings  of 


a  Texas  state  committee  as  follows: 


The  criteria  selected  are  classified  under  seven  major 
headings:  (1)  'The  Children',  (2)  'The  Teachers',  (3)  'The  Program 
of  School  Life',  (h)  'Instructional  Equipment  and  Supplies',  (5) 
'Administration'.    Each  major  heading  is  subdivided  into  topics 
representing  significant  phases  of  the  area  designated. 


y 


Wrightstone    in  his  article  on  "Trends  in  Evaluation"  points  out 


that: 


Modern  evaluative  criteria  is  characterized  by  several 
criteria. 

1)  It  attempts  to  measure  a  comprehensive  range  of  objec- 
tives of  the  modern  school  curriculum  rather  than  limited  subject- 
matter  achievement  only. 

2)  Modern  evaluation  uses  a  variety  of  techniques  of 
appraisal  such  as — achievement,  attitude,  personality  and  character 
tests.    Also  rating  scales  of  products,  interviews,  controlled - 
observation  techniques,  sociometric  techniques  and  anecdotal 
records. 

3)  Evaluation  includes  integrating  and  interpretating  the 
various  indexes  of  behavior  changes  so  as  to  construct  an  inclusive 
portrait  of  an  individual  or  an  educational  situation. 

"Evaluation  is  a  constant  process  of  the  consideration  of  the 
3/ 

degree,"  says  Ostrander  ,  "to  which  the  results  and  by-products  of  the 


education  program  reach  the  stated  and  implied  objectives  of  the 


program. 


ii 


Results  are  often  indecisive  and  it  has  been  concluded  that 


evaluation  is  not  completely  a  precise,  scientific  process — defined  by 


tables,  statistics,  infallible  instruments,  and  measureables. 


1/  N.  B.  Henry,  "Evaluating  Elementary  Schools,"  The  Elementary  School 
Journal,  (February,  19l|6)  U6:307-308.  p.308. 

2/  J.  W.  Wrights tone,  op.  cit.,  p. 91. 

3/  R.  H.  Ostrander,  op.  cit.,  p. 87. 
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1/ 

Vincent    points  out  that  the  direct  evaluation  of  the  results  we  are 
seeking  in  education  is  discouraging  because  they  take  so  long  to 
mature.    Direct  measurement  would  be  difficult  if  not  impossible  to 
obtain  so  we  must  seek  techniques  to  "measure  them  indirectly." 

In  the  case  of  the  unavailability  of  the  direct  interview  method, 
a  system  whereby  this  can  be  done  by  practical  means  must  be  devised. 
The  most  popular*  and  most  practical  method,  from  an  economical  and  time 
saving  point  of  view,  must  be  used. 

u 

RESEARCH  PERTAINING  TO  THE  INQUIRY  FORM.-  -Nichols    defines  a 

questionnaire  as: 

A  questionnaire  is  the  outward  expression  of  an  inner  urge 
to  find  out  how  to  do  a  better  piece  of  work:  how  to  teach  some- 
thing better;  how  to  find  new  instruction  material;  how  to  be  sure 
current  procedures  are  sound;  how  to  remedy  faults  of  technique. 
It  may,  and  often  does,  result  from  a  yearning  for  corroborating 
evidence  as  to  the  soundness  of  the  things  we  believe.    It  is  a 
neat  device  for  use  in  gathering  factual  data. 

With  only  a  cursory  glance  at  this  statement,  the  reader  probably 

dismisses  this  statement  from  his  mind,  thinking  it  too  obvious  to 

ponder  over.    A  little  thought  would  convince  him  that  this  is  not  just 

another  empty,  turgid,  abstruse  conjecture  on  the  part  of  Nichols,  but 

couched  within  the  language  is  the  framework  of  the  true  definition  of 


1/  William  S.  Vincent,  "Talcing  Inventory — Finding  Ways  to  Evaluate 
Results,"  School  ^ecutive,  (January,  19^0 )  69:73-73. 

2/  Frederick  G.  Nichols,  "Criticism,  Comment  and  Challenge,"  Journal  of 
Business  Education,  (June,  1933)  8:8.  p. 8. 


1/ 

a  questionnaire.    Huffman    elaborates  on  the  concept  of  the  questionnaire 

when  he  states: 

Since  the  questionnaire  is  not  only  a  means  for  determining 
the  extent  of  agreement  in  observations,  but  it  is  also  a  trail- 
blazer  in  fundamental  educational  investigation,  it  is  an  instru- 
ment well  deserving  of  meticulous  construction.    Suggested  for 
improving  the  questionnaire  so  that  it  will  attain  its  optimum 
usefulness  as  a  tool  of  research  are:  finding  its  weaknesses  and 
strengths,  studying  the  principles  of  language  uhich  make  a  question 
answerable,  and  following  a  scientific  procedure  in  the  development 
of  the  questionnaire  itself. 

There  are  several  concepts  that  are  fundamental  to  the  formulation 

and  construction  of  311  answerable  question.    "The  formulation  of  an 

V 

answerable  question,"  says  Huffman  ,  "is  fundamental  to  the  scientific 
method.    Research  workers  are  confronted  with  the  problem  of  formulating 
questions,  for  oftentimes  the  question  technic  is  requisite  to  making 
observations  which  lead  to  the  answers  of  their  original  question." 

There  are  many  and  diverse  factors  that  aid  in  the  development 
of  a  good  questionnaire.    The  Research  Division  of  the  National  Educa- 
tion  Association    is  of  the  belief  that,  "Every  effort  should  be  made 
to  reduce  the  writing  necessary  to  answer  the  questionnaire.    It  should 
be  worded  so  that  the  answers  may  be  expressed  by  yes  or  no,  check  marks, 
underlining  or  encircling,  by  figures,  or  by  short  answers  of  a  few 
words . " 


1/  Harry  Huffman,  "Improving  The  Questionnaire  As  A  Tool  of  Research," 
The  National  Business  Education  Quarterly,  (October,  19U8)  17:15-18  & 
p.  60. 

2/  Ibid. ,  p. 18. 


3/  Research  Division,  National  Education  Association,  "The  Questionnaire," 
National  Education  Association  Research  Bulletin,  (Bulletin  8,  1930)  p. 21. 


Concurring  in  this  observation  are  Davis  and  Barrow.  They  are 
of  the  belief  that,  "The  majority  of  questionnaires  should  be  made  so 
constructed  that  answers  could  be  made  by  checking  the  correct  word, 
phrases  or  number. " 

The  construction  of  the  questionnaire  can  influence  the  answer 

or  answers  that  will  be  given.    Benjamin    says  that,  "It  is  generally 

recognized  that,  when  multiple  choice  questionnaires  are  employed,  bias 

in  response  can  result  from  the  particular  order  in  which  the  options 

are  arranged.    To  neutralize  such  bias,  several  forms  of  the  same  question 

are  frequently  prepared,  differing  only  in  the  order  of  presentation  of 

the  questions  and  of  the  optional  answers  for  each  question. " 

There  are  varied  means  of  having  the  respondent  answer  a  ques- 

3/ 

tionnaire.    In  addition  to  those  listed  above  Koos    maintains  that,  "A 
type  of  response  not  unlike  the  yes-or-no  type  in  its  simplicity  is  that 
of  checking  the  item  or  items  in  a  series  which  best  describe  a  practice 
or  best  express  one's  preference.    However,  it  tends  to  correct  the 
deficiencies  of  the  yes-or-no  type  by  opening-up  a  wider  array  of  alter- 
natives from  which  to  select  and  at  the  same  time  avoiding  restriction 
to  positive  and  negative  responses."    One  of  the  advantages  of  the 


1/  R.  A.  Davis  and  S.  L.  Barrow,  "Critical  Study  of  the  Questionnaire 
in  Education,"  Education  Administrator  and  Supervisor,  (February,  1935) 
21:137-HUi.  p.lIOT 

2/  K.  Benjamin,  "Combining  Responses  on  Two  Forms  of  a  Questionnaire 
with  Options  in  Inverse  Order,"  Public  Opinion  Quarterly,  (No.  13,  19U9) 
h: 688-690.  p. 688. 

3/  Leonard  V.  Koos,  The  Questionnaire  in  Education,  The  MacMillan  Company, 
New  York,  1928.  pp. 82-83. 
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checking  type  of  responses  over  some  of  the  other  types  is  that  it  aids 
in  reducing  to  negligible  proportions  the  amount  of  writing  that  it 

il 

reouires  of  the  respondents. 
2/ 

Koos    has  said  that,  "One  may  safely  conclude  that  while  ques- 
tionnaires should  be  as  brief  as  possible,  such  factors  as  timeliness, 
merit  of  the  study,  and  adequate  motivation  will  often  offset  the  dis- 
advantages of  length."    However,  the  Research  Division  of  the  National 

3/ 

Education  Association    is  adamant  in  its  belief  that,  "Short  question- 
naires tend  to  obtain  a  higher  percent  of  return  than  long  ones." 

The  major  purpose  of  constructing  a  questionnaire  is  to  obtain 

y 

valid  results.     "Validity,"  says  Parry,     "is  basic  to  all  research  and 

y 

the  concept  must  be  made  more  specific."    Koos    says  that,  "in  question- 
naire studies  we  must  be  concerned  with  the  proportion  of  response  from 
those  who  are  approached  ...  We  are  still  far  from  knowing  in  detail 
the  proportions  of  responses  required  to  afford  a  given  degree  of 
validity  to  the  findings  of  our  investigations." 

In  the  attempt  to  abbreviate  to  a  few  phrases  the  apparently 
important  elements  in  obtaining  high  returns,  the  following  statements 
in  approximately  a  descending  order  of  importance  are  given: 


1/  Leonard  V.  Koos,  op.  cit.,  p.l2i|. 
2/  Ibid.,  pp.  131-132. 

3/  Research  Division,  National  Mucation  Association,  op.  cit.,  pp.  19-20. 

k/  Hugh  J.  Parry  and  Helen  M.  Cross  ley,  "Validity  of  Response  to  Survey 
Questions,"  Public  Opinion  quarterly,  (No.  lit,  1950)  1:61-80.  p.6l. 

$/  Leonard  V.  Koos,  op.  cit.,  p. 132. 


1.  The  questionnaire  should  be  sent  to  people  who  are  aware  of 
your  professional  repute  or  who  will  feel  some  personal  obligation  to 
reply. 

2.  The  best  possible  technique  in  writing  the  questionnaire  should 
be  utilized. 

3.  Construct  the  questionnaire  in  such  a  manner  that  the  res- 
pondent will  find  it  easy  to  reply. 

U.    Use  objective  unequivocal  but  sensible  questions.    Do  not 
avoid  using  written  answers  but  be  wary  of  "essay  answers." 

5.    Imply  advisedly  such  incidental  pressures  as  "moral  obliga- 

y 

tion  to  reply." 

In  addition  to  the  above  prerequisites  for  obtaining  high 
returns  are: 

1.  We  should  avoid  using  abstractions  or  concept  words  for  which 
philosophers  require  definition  before  they  begin  to  argue. 

2.  The  chief  protections  against  a  meaningless  mess  are:  Use 

of  concrete  words,  constant  awareness  of  the  problem,  inclusion  of  more 

2/ 

than  one  auestion,  and  strict  attention  to  brevity  and  simplicity. 
'3/ 

Reid    says,  "There  is  the  obvious  importance  of  preparing  a 


1/  Herbert  A.  Troops,  "Predicting  the  Returns  from  Questionnaires:  A 
Study  in  the  Utilization  of  Qualitative  Data,"  Journal  of  Experimental 
Education,  (March,  193S)  3:20l|-2l£. 

2/  Stanley  L.  Payne,  "Thoughts  about  Meaningless  Questions,"  Public 
Opinion  Quarterly,  (No.  lU,  1950)  U: 687-696. 

3/  S.  Reid,  "Respondents  and  Non-respondents  to  Mail  Questionnaires," 
Educational  Research  Bulletin,  (April,  19U2)  21:87-96.  p. 96. 


questionnaire  carefully  and  attractively  so  as  to  bring  in  maximum 
returns.    Everything  possible  in  the  way  of  typography,  phraseology, 
and  salesmanship  should  be  done  to  increase  the  number  of  original 

y 

respondents."    In  addition  to  these  beliefs  Reid    thinks  that  the  amount 
of  returns  are  due  to  many  other  apparently  insignificant  factors  such 
as,  "the  type  of  individuals  to  whom  questionnaires  were  sent,  the  pres- 
tige of  the  investigator,  the  length  of  the  questionnaire,  and  the  ease 
with  which  it  can  be  answered." 

Many  gratuitous  overtures  have  to  be  made  to  entice  the  respon- 
dents  to  return  the  questionnaires  and  Price    believes  strongly  that 
return  envelopes  are  necessary.    He  says,  "In  sending  out  questionnaire 
through  the  mail  most  research  workers  have  never  doubted  the  utility 
of  placing  a  stamp  on  the  return  envelope  in  order  to  increase  the 
proportions  of  returns." 

Some  of  the  more  salient  points  to  be  remembered  in  constructing 
the  inquiry  form  are: 

1.  The  directions  should  be  clearly  stated  and  complete. 

2.  The  questions  should  not,  under  any  dire  urns  tances,  be  com- 
plicated. 

3»    Questions  that  are  of  a  similar  nature  should  be  grouped  in 
order  to  reduce  the  need  for  repetition  of  directions. 

U.    Each  question  should  be  evaluated  carefully  in  light  of  the 

1/  S.  Reid,  op.  cit.,  p. 87. 

2/  D.  0.  Price,  "On  the  use  of  Stamped  Return  Envelopes  with  Mail  Ques- 
wionnaires,"  American  Social  Research,  (October,  195>0)  15:672.  p. 672. 


specific  purpose  it  is  to  fulfill. 

5.  Each  question  should  be  stated  in  such  a  manner  that  it  will 
be  conducive  to  uniformity  of  interpretation. 

6.  Each  question  should  be  worded  and  arranged  so  as  to  promote 

y 

ease  and,  above  all,  accuracy  in  the  tabulation  of  data. 

u 

"Conciseness,"  declares  Romine    ,  "consistent  -with  clarity  and 
completeness  is  the  keynote." 

Letters  of  Transmittal: 

The  statement  of  the  purpose  of  the  questionnaire  can  be  given 

in  two  ways  according  to  the  Research  Division  of  the  National  Educa- 

3/ 

tion  Association.    The  Research  Division    says,  "The  statement  of  pur- 
pose may  be  given  in  a  letter  of  transmittal  or  in  an  introductory 
paragraph  on  the  questionnaire.    It  should  be  brief,  but  adequate,  and 
should  not  be  omitted."    These  tvo  alternatives  are  given  to  the  person 
constructing  the  questionnaire  to  work  with.    Each  alternative  should 
be  judged  in  light  of  the  purpose  that  it  is  to  serve.    The  Research 

y 

Division     continues  its  explanation  of  transmittal  letters  by  saying: 

A  separate  letter  of  transmittal  is  usually  desirable.  It 
offers  an  opportunity  to  explain  the  purpose  and  extent  of  the  in- 
vestigation and  to  give  other  pertinent  information.    It  should 
be  brief  and  to  the  point.    Such  a  letter  may  be  addressed  in 


1/  S.  A.  Romine,  "Criteria  for  a  Better  Questionnaire,"  Journal  of 
Educational  Research,  (September,  1°1;8)  U2:69-71. 

2/  Ibid.,  p. 71. 

3/  Research  Division,  National  Education  Association,  op.  cit. ,  p. 23. 
h/  Ibid.,  p. 23. 


general  terms  •  •  •  Still  better  the  name  and  address  of  the  in- 
dividual receiving  the  questionnaire  may  be  typewritten  on  a  multi- 
graphed  form. 

1/ 

Moore     is  of  the  opinion  that,  "The  typewritten  letter  of  trans- 
mittal is  more  effective  in  getting  questionnaire  returns  than  the 
duplicated  letter  of  transmittal." 

The  content  of  the  letter  of  transmittal  has  to  be  carefully 
planned.    It  cannot  be  relegated  to  a  place  of  minor  importance  as  Koos 
points  out,  "The  character  of  letters  accompanying  questionnaires  and 
prepared  by  the  investigator  is  of  much  more  moment  than  many  novices 
in  questionnaire  investigation  are  wont  to  believe.    These  letters 
should  not  only  explain  the  project  sufficient  to  make  clear  the  purport 
of  the  study  but  should  be  so  framed  as  to  motivate  the  recipient  to 
respond." 


1/  Clarence  C.  Moore,  "Increasing  the  Returns  from  Questionnaires," 
Journal  of  Educational  Research,  (October,  lQUl)  35>:138-lkl. 


2/  Leonard  V.  Koos,  op.cit. ,  p.l2U 


CHAPTER  III 
PLAN  OF  THE  STUDY 


CHAPTER  III 
PLAN  OF  THE  STUDY 


The  purpose  of  this  research  project  was  to  determine  exactly 
and  specifically  whether  the  "Elementary  Evaluative  Criteria"  is  doing 
what  it  purports  to  do. 

The  writers  met  with  the  author  to  discuss  the  advisability  of 
attempting  to  evaluate  the  instrument.  The  decision  was  reached  that 
it  was  a  worthwhile  project  and  had  much  merit  behind  it. 

The  writers  went  about  the  task  of  making  an  outline  of  proce- 
dures to  be  followed  in  the  attempt  to  arrive  at  the  best  possible  and 
feasible  method  of  carrying  through  this  research  effort. 

The  voucher  slips  of  all  copies  of  the  evaluative  instrument 
that  had  been  purchased  were  made  available  to  this  group.    It  was 
decided  that  the  purchasers  and  persons  using  the  criteria  were  the 
logical  people  to  be  queried  as  to  the  practicality  of  the  criteria. 
It  was  found  that  these  purchasers  were  located  in  many  sections  of 
the  country.    In  light  of  this ,  the  writers  reviewed  research  to  ascer- 
tain the  best  possible  method  of  obtaining  valid  responses  to  the  ques- 
tions that  had  to  be  asked.    The  method  decided  upon  was  the  mail  ques- 
tionnaire. 

In  conjunction  with  this,  research  pertaining  to  evaluative 
criteria  was  reviewed.    The  specific  conclusions  that  were  drawn  from 
this  research  are: 

1.    Modern  evaluation  uses  a  variety  of  techniques  of  appraisal. 
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2.  Evaluative  criteria  should  attempt  to  measure  a  comprehen- 
sive range  of  objectives  rather  than  limited  objectives. 

5.  The  questionnaire  is  a  valid  means  of  measuring  evaluative 
criteria  in  any  case  where  the  respondent  is  not  readily  accessible  to 
the  inquirer* 

The  Development  of  the  Questionnaire, 

In  order  that  the  evaluation  of  the  "Elementary  Evaluative 
Criteria"  would  be  reliable  and  valid,  a  questionnaire  was  devised  to 
obtain  responses  to  the  information  requested.    The  questionnaire  was 
revised  several  times  before  a  satisfactory  instrument  was  formed. 

Description  of  the  Questionnaire  Used  in  This  Study. 

The  questionnaire  form  called  for  three  general  types  of  infor- 
mation, namely:  a  section  on  the  various  uses  to  which  the  "Elementary 
Evaluative  Criteria"  were  subjected,  a  section  on  the  suitability  of  the 
contents  of  the  "Elementary  Evaluative  Criteria"  in  light  of  the  res- 
pondent's personal  convictions,  and  a  section  on  general  reactions, 
which  included  a  space  for  any  suggestions  or  comments  that  the  res- 
pondents cared  to  make. 

The  questions  that  pertained  to  the  section  on  the  various  uses 
to  which  the  criteria  were  subjected  are  to  be  found  in  Appendix  A. 

The  questions  that  pertained  to  the  section  on  the  suitability 
of  the  contents  of  the  criteria  are  to  be  found  in  Appendix  3. 

The  questions  that  pertained  to  the  section  on  the  general 
reactions  are  to  be  found  in  Appendix  C. 
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Description  of  the  Letter  of  Transmittal, 

The  writers  decided  that  a  separate  letter  of  transmittal  ex- 
plaining the  purpose  of  the  study  was  more  appropriate  than  a  separate 
paragraph  on  the  questionnaire  itself.    The  letter  of  transmittal  was 
multigraphed.    A  space  was  left  for  the  heading  and  address  to  be  type- 
written on  the  letter.    Several  revisions  of  this  letter  were  made  be- 
fore an  acceptable  form  was  completed.    A  copy  of  this  form  letter  is 
found  in  Appendix  D, 

Description  of  Practices  Utilized  to  Increase  Returns, 
It  was  realized  that  the  greater  the  number  of  responses  received, 
the  more  valid  the  study  would  be.    A  method  had  to  be  devised  to  en- 
courage the  respondents  to  reply.    The  questionnaire  was  made  as  short 
as  practicable.    A  copy  of  the  complete  questionnaire  is  found  in 
Appendix  E. 

A  copy  of  the  questionnaire,  found  in  Appendix  E,  and  a  copy  of 
the  letter  of  transmittal,  found  in  Appendix  D,  were  to  be  sent  to  each 
person  who  was  to  be  questioned.    The  writers  decided  that  it  would 
increase  results  to  enclose  a  stamped,  self -addressed  envelope  for  the 
convenience  of  the  respondents. 

Sample  of  the  Study. 

The  writers  did  not  use  selective  sampling  because  a  questionnaire 
was  sent  to  each  person  who  had  purchased  a  copy  or  copies  of  the 
criteria  directly  from  the  publisher,  regardless  of  the  purchase  date 
or  the  position  of  the  purchaser.    This,  in  itself,  would  indicate  that 


low  returns  were  to  be  expected  due  to  the  fact  that  many  of  the 
purchasers  had  no  opportunity  to  make  use  of  the  criteria  and  therefore 
made  no  reply. 

Distribution  of  the  Questionnaire* 

The  questionnaire,  accompanied  by  the  material  listed  above,  was 
mailed  to  U93  purchasers.    Of  the  U93  purchasers,  168  had  purchased  five 
or  more  copies.    The  purchasers  are  to  be  found  in  many  sections  of  the 
country. 

Percentage  of  Returns. 

A  total  of  215  answers  were  received  by  the  deadline  that  was 
established  by  the  writers.    The  deadline  was  set  as  February  10,  195>5>« 
The  first  questionnaires  were  sent  out  on  November  17,  19£U.  February 
10,  19^5  was  set  as  a  deadline  in  order  that  the  writers  could  tabulate 
the  results  in  time  to  complete  the  study. 

Handling  the  Returns. 

A  mailing  list  was  kept  by  the  writers  of  all  questionnaires  that 
were  sent  out.    As  each  questionnaire  was  returned,  it  was  checked  off 
the  mailing  list  and  recorded  on  a  separate  sheet.    A  master  chart  was 
set  up  to  tabulate  the  results  of  the  responses  to  the  questionnaires. 
As  these  responses  were  tabulated,  an  indication  was  made  on  the  returned 
questionnaire  to  indicate  that  the  information  had  been  included  on  the 
master  chart. 
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THE  ANALYSIS  OF  DATA 

The  questionnaire,  described  in  Chapter  III  and  the  Appendices, 
was  built  for  purposes  of  inventory  and  survey.    It  was  designed  to 
elicit  valid  responses  as  to  the  effectiveness  of  "The  Elementary  Eval- 
uative Criteria"  as  an  evaluating  instrument.    An  analysis  of  these  data 
centers  around  the  responses  that  the  respondents  indicated.  This 
survey  was  also  conducted  for  the  purpose  of  attempting  to  find  out  if 
any  particular  section  of  the  criteria  needed  revision.    Some  of  the 
respondents  made  comments  regarding  their  suggested  revisions.  These^ 
while  not  statistically  accurate,  are  believed  to  be  very  important  and 
are  included  in  the  analysis  of  data. 

The  data  have  been  analyzed  in  four  ways.    The  questionnaires 
were  sent  to  persons  purchasing  one  to  four  copies,  and  persons  pur- 
chasing five  or  more  copies.    It  was  believed  that  it  would  be  more 
meaningful  to  analyze  the  data  as  follows: 

1)  An  analysis  was  made  of  all  replies. 

2)  A  separate  analysis  was  made  of  those  persons  purchasing  five 
or  more  copies. 

5)  An  analysis  was  made  of  general  comments  from  persons  pur- 
chasing one  to  four  copies  of  the  criteria. 

4)  An  analysis  was  made  of  general  comments  from  persons  pur- 
chasing five  or  more  copies  of  the  criteria. 

-IP- 
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Tables  I  through  "VI  are  computed  on  the  basis  of  215  responses. 
These  comprise  the  total  number  obtained  in  answer  to  the  questionnaire. 

Tables  VII  through  XI  are  computed  on  the  basis  of  seventy-nine 
replies.    These  comprise  all  respondents  who  purchased  five  or  more 
copies  of  the  criteria. 
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TABLE  I 

PURPOSES  FOR  WHICH  THE  CRITERIA  WERE  USED 


Type  of  Use 

Number 

Per  Cent 

Evaluation  of  a  total  school  system 

31 

Ub.Ul 

Evaluation  of  a  single  school 

51 

23.72 

Evaluation  of  one  phase  of  the  program 

60 

27.90 

Basis  for  discussion  at  teacher's  meetings 

93 

U3.2* 

Study  groups,  resources,  or  reference  materials 

93 

U3.2S 

No  use  designated 

22 

10.23 

Table  I  indicates  that  twenty-two  respondents  or  10.23  per  cent 
of  the  total  number  of  replies  received  made  no  response  as  to  the  use 
they  subjected  the  criteria.    Thirty-one  persons  or  li+.l+l  per  cent  used 
the  criteria  to  evaluate  a  total  school  system.    The  majority  of  these 
thirty-one  persons  belonged  to  the  group  who  had  purchased  five  or  more 
copies  of  the  criteria.    Ninety-three  persons  or  U3.2$  per  cent  of  the 
total  group  used  the  criteria  as  a  basis  for  discussion  at  teacher's, 
meetings.    The  same  number  applies  as  to  its  use  in  study  groups,  as 
resources,  or  reference  materials.    Many  of  the  respondents  used  the 
criteria  for  more  than  one  purpose  and  so  indicated  by  checking  in  the 
appropriate  places. 

Table  II  indicates  the  responses  that  were  made  as  to  the  suit- 
ability or  unsuitability  of  the  criteria.    Negligible  responses  were 
made  pertaining  to  the  deletion  of  any  section  of  the  criteria.  Kinder- 
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TABLE  II 

RATINGS  DESIGNATED  TO  VARIOUS  SECTIONS  OF  THE  CRITERIA 


Satisfactory     Needs  Delete        No  Comment 

Revision 

Sections  of 


Criteria 

No. 

% 

No* 

at 

No. 

of 

No. 

of 

Manual 

oo 

1*0.00 

3 

l.yO 

0 

•  00 

12o 

58. oO 

Pnilosopny  ol 

Objectives 

82 

38.13 

8 

3.72 

0 

.00 

125 

58.13 

Kindergarten 

79 

36.71* 

5 

2.32 

2 

.93 

129 

60.00 

Arithmetic 

83 

38.60 

5 

2.32 

1 

.16 

126 

58.60 

Arts  &  Crafts 

81 

37.67 

10 

i*.65 

0 

.00 

121* 

57.67 

Health  &  Physical 

Education 

76 

35.31* 

9 

1*.18 

0 

.00 

120 

55.81 

Language  Arts 

82 

38.13 

9 

1*.18 

0 

.00 

121* 

57.67 

Music 

79 

36.71* 

i» 

1.86 

0 

.00 

132 

61.39 

Science 

83 

38.60 

2 

.93 

0 

.00 

130 

6o.l*6 

Social  Studies 

78 

36.27 

7 

3.25 

1 

.1*6 

129 

60.00 

Library  Services 

78 

36.27 

5 

2.32 

1 

.1*6 

131 

60.93 

Guidance  Services 

78 

36.27 

9 

1*.18 

2 

.93 

126 

58.6o 

School  Plant 

80 

37.20 

I* 

1.86 

0 

.00 

131 

60.93 

School  Staff  & 

Administration 

80 

37.20 

6 

2.79 

0 

.00 

129 

60.00 

Individual  Staff 

Member  Data  Sheet 

78 

36.27 

5 

2.32 

1 

.1*6 

131 

60.93 

23 


garten  and  guidance  services  sections  received  the  highest  per  cent  of 
responses  as  pertaining  to  deletion  and  this  percentage  fails  to  reach 
one  per  cent.    The  comments  that  accompanied  the  returns  indicated  to 
the  writers  that  the  persons  who  advocated  deletions,  in  most  cases, 
were  those  who  did  not  have  that  particular  phase  of  the  criteria 
included  in  their  school  program. 

The  "no  comment"  section  leads  in  percentage.    This  was  to  be 
expected  due  to  the  fact  that  many  of  the  respondents  used  the  criteria 
for  study  groups  and  other  related  projects.    (See  Table  I)    The  respon- 
dents who  used  the  criteria  in  study  groups,  etc. ,  had  no  valid  means 
of  judging  the  criteria  and  so  indicated  in  their  written  comments. 

The  respondents  who  indicated  that  the  criteria  needed  revision 
made  written  comment,  in  some  cases,  as  to  the  amount  and  type  of 
revision  that  needed  to  be  done.    In  all  cases  where  a  need  of  revision 
is  indicated  the  percentage  is  very  low. 

The  percentage  of  respondents  who  made  an  indication  as  to  the 
suitability  of  the  criteria  is  very  high  if  the  "no  comments"  are  dis- 
carded. 


TABLE  III 
GENERAL  REACTIONS  TO  CRITERIA 

A.  Adequacy  of  instructions  for        B#  Needs  further  integration  and 
rating  checklists  and  eval-  coordination 
uations    

Adequate  Inadequate  Yes  No 

Number    Per  Cent     Number    Per  Cent     Number    Per  Cent     Number    Per  Cent 

105      U8.83  11         5.11  h9        22.79  31  Hi.Ul 

Table  III  begins  the  analysis  of  data  pertaining  to  Section  III 
of  the  questionnaire.  (See  Appendix  C)  Table  III  is  divided  into  two 
parts,  A  and  B. 

In  part  A,  ll6  respondents  made  a  reply  as  to  the  adequacy  of 
the  instruction  for  rating  checklists  and  evaluation,    105  or  U8.83  per 
cent  stated  that  the  instructions  were  adequate,  while  11  persons  or 
5.11  per  cent  stated  that  the  instructions  were  inadequate.  Ninety-nine 
persons  made  no  comment. 

Part  B  concerns  itself  with  the  need  for  further  integration  and 
coordination  between  the  various  subject  areas  found  in  the  criteria. 
Forty-nine  persons  or  22.79  per  cent  stated  that  further  integration  and 
coordination  between  the  subject  areas  was  needed.    Thirty-one  or  1U.U1 
per  cent  stated  that  there  was  no  need  for  further  integration  and 
coordination  between  subject  areas. 


TABLE  IV 

POSITION  OF  INDIVIDUAL  FILLING  OUT  THE  QUESTIONNAIRE 


Type  of  Position 

Number 

Per  Cent 

Administrator 

1U8 

68.83 

Supervisor 

19 

8.83 

Teacher 

6 

2.79 

No  Comment 

m 

19.53 

Table  IV  is  an  indication  of  the  positions  held  by  the  respondents. 
The  highest  per  cent  of  returns  came  from  administrators.    This  group, 
administrators,  is  comprised  of  principals,  superintendents,  and,  in 
some  cases,  members  of  boards  of  education. 


TABLE  V 

USE  OF  THE  CRITERIA  WITH  LAY  PERSONNEL 


Number  of  communities  using  criteria  with  laymen 

Per  Cent 

29 

13.U8 

Table  V  indicates  the  number  and  percentage  of  the  total  number 
of  communities  using  the  criteria  with  laymen.  This  number  may  not  be 
the  total  number  of  communities  using  the  criteria  with  laymen,  but  it 


does  indicate  the  number  of  respondents  who  indicated  on  their  question- 
naire that  they  had  used  the  criteria  with  laymen. 
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TABLE  VI 

IMPROVEMENT  INDICATED  IN  COMMUNITIES  USING  CRITERIA 


Degree  of  Improvement 


Number 


Per  Cent 


Extensive 


8 


3.72 


Some 


72 


33. U8 


None 


k 


1.86 


Table  VI  is  an  indication  of  the  degree  of  improvement  in  the 
school  systems  of  eighty-four  respondents.    The  remaining  130  respondents 
were  noncommittal  as  to  the  extent  of  improvement  in  their  school 
systems.    Of  the  eighty-four  replies  to  this  question,  seventy- two  or 
33. U8  per  cent  stated  that  the  use  of  the  criteria  had  brought  about 
some  improvement. 

Tables  VII  through  XI  consist  of  an  analysis  of  data  as  computed 
on  the  basis  of  responses  from  persons  who  purchased  five  or  more  copies 
of  the  "Elementary  Evaluative  Criteria." 
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TABLE  VII 

RATINGS  DESIGNATED  TO  VARIOUS  SECTIONS  OF  THE  CRITERIA 


Satisfactory     Needs  Delete        No  Comment 

Revision 

Sections  of 


Criteria 

.NO. 

<£ 
fi 

MA 

HO  . 

)» 

NO* 

riflnucLL 

*y 

7.f\  7n 
.50  •  fU 

c 

U 

•  uu 

lift 

40 

An 

rnjJLOSopny  fit 
Objectives 

30 

37.97 

1 

1.26 

0 

.00 

U8 

60*75 

Kindergarten 

26 

32.91 

4 

5.06 

1 

1.26 

48 

60.75 

Arithmetic 

32 

Uo.5o 

3 

3.79 

0 

.00 

44 

55.69 

Arts  &  Crafts 

27 

3U.17 

5 

6.32 

0 

.00 

U7 

59.49 

Health  &  Physical 
Education 

nr' 

25 

31.&4 

7 

o.oo 

0 

•  00 

U7 

59.49 

Language  Arts 

30 

37.97 

5 

0.32 

0 

•  00 

I.I 

55.09 

Music 

29 

36.70 

2 

2.53 

0 

.00 

U8 

60.75 

Science 

30 

37.97 

1 

1.26 

0 

•00 

U8 

60.75 

Social  Studies 

28 

35.1* 

6.32 

1 

1.26 

46 

58.22 

Library  Services 

26 

32.91 

3 

3.79 

1 

1.26 

U9 

62.02 

Guidance  Services 

25 

3l.6b 

6 

7.59 

2 

2.53 

U6 

58.22 

School  Plant 

30 

37.97 

2 

2.53 

0 

.00 

U7 

59.U9 

School  Staff  & 
Admini  s tr ation 

29 

36.70 

U 

5.06 

0 

.00 

46 

58.22 

Individual  Staff 
Member  Data  Sheet 

28 

35.Ub 

3 

3.79 

1 

1.26 

U7 

59.49 

IIV 


MB 


lo 


Table  VII  indicates  the  responses  that  were  made  as  to  the  suit- 
ability or  unsuitability  of  the  criteria.    This  table  should  be  compared 
to  Table  II  to  obtain  an  over-all  picture.    This  table  indicates  that 
negligible  responses  were  made  pertaining  to  the  deletion  of  any  section 
of  the  criteria,    When  compared  with  Table  II  the  reader  will  notice 
that  almost  all  of  the  recommendations  for  deletion  came  from  the  group 
that  purchased  five  or  more  copies  of  the  criteria. 

The  "No  Comment"  section  is  high  in  number  and  percentage.  The 
reason  for  this  is  the  same  as  is  applicable  to  Table  II.    (See  Table 
II  and  explanation). 

When  compared  to  Table  II  the  percentage  of  respondents  indicating 
that  the  criteria  needs  revision  in  certain  areas  is  slightly  higher  in 
Table  VII.    This  may  be  significant.    The  percentage  has  not  increased 
to  a  marked  degree  in  any  specific  area  but  it  would  be  worthwhile  to 
note  the  correlation  between  Table  II  and  Table  VII. 

The  percentage  of  respondents  who  made  an  indication  as  to  the 
suitability  of  the  criteria  is  very  high,  as  is  the  case  in  Table  II, 
if  the  "No  Comment"  section  is  deleted. 
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TABLE  VIII 
GENERAL  REACTIONS  TO  CRITERIA 

A.  Adequacy  of  instructions  for        B.  Needs  further  integration  and 
rating  checklists  and  eval-  coordination 
uations  

Adequate  Inadequate  Yes  No 

Number   Per  Cent     Number    Per  Cent     Number   Per  Cent     Number    Per  Cent 

la        51.89  2  2.53  15        18.98  17  21.51 

Table  VIII  should  be  compared  with  Table  III. 

Part  A.    Forty-one  respondents  or  5l»89  per  cent  of  the  persons 
who  purchased  five  or  more  copies  of  the  criteria  stated  that  the 
instructions  for  rating  checklists  and  evaluation  were  adequate.  Two 
persons  or  2.53  per  cent  of  this  group  stated  that  the  instructions  were 
inadequate. 

Part  B.    Fifteen  persons  or  18.98  per  cent  stated  that  the  criteria 
need  further  integration  and  coordination.    Seventeen  persons  or  21.51 
per  cent  of  the  group  stated  that  there  was  no  need  for  further  integra- 
tion and  coordination. 
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TABLE  IX 

POSITION  OF  INDIVIDUAL  FILLING  OUT  THE  QUESTIONNAIRE 


Type  of  Position  Number       Per  Cent 

Administrator  U3 

Supervisor  7  8.86 

Teacher  0  .00 

No  Comment  29  36.70 

Table  IX  is  an  indication  of  the  positions  held  by  the  respondents 
who  purchased  five  or  more  copies.    (For  comparative  purposes  see  Table 
IV)    As  in  Table  IV  the  highest  per  cent  of  returns  came  from  the  admin- 
istrators.   This  group  is  comprised  of  principals,  superintendents,  and 
in  some  cases,  members  of  boards  of  education. 


TABLE  X 

USE  OF  THE  CRITERIA  WITH  LAY  PERSONNEL 


Number  of  communities  using  criteria  with  laymen 

Per  Cent 

9 

11.39 

Table  X  indicates  that  only  nine  communities  used  the  criteria 
with  laymen  participating  in  its  use.    Table  V,  and  Table  X  indicate  the 
number  of  respondents  who  indicated  on  their  questionnaire  that  they 
had  used  the  criteria  with  laymen. 

The  remaining  respondents  made  no  indication  whether  they  used 
the  criteria  with  laymen  or  not. 


TABLE  XI 

IMPROVEMENT  INDICATED  IN  COMMUNITIES  USING  CRITERIA 


Degree  of  Improvement 

Nmber 

Per  Cent 

Extensive 

u 

5.63 

Some 

31 

39. 2h 

None 

k 

5.63 

Table  XI  is  based  on  seventy-nine  replies.    Thirty-nine  respon- 
dents answered  this  question.    The  majority  of  those  that  answered 
stated  that  the  use  of  the  criteria  brought  about  some  improvement  in 
their  school  systems. 


Analysis  of  Comments. 

The  persons  who  purchased  one  to  four  copies  made  several  general 
comments  that  should  be  considered  in  the  analysis  of  data.    The  follow- 
ing comments  are  obtained  from  the  written  comments  that  were  directed 
to  the  writers  in  answer  to  their  request  for  comments  concerning  the 
criteria: 

General  Nature  Comments. 

"Elementary  Evaluative  Criteria  has  been  used  by  a  limited  number 
of  teachers.    We  are  planning  to  use  the  material  more  extensively  later 
this  year  and  next  year.    Those  who  have  studied  the  criteria  feel  that 
it  will  be  very  valuable  to  help  us  in  the  improvement  of  our  work.  It 
is  excellent." 

"My  comments  are  very  general.    I  believe  your  criteria  make  it 
hard  to  evaluate  a  school  or  system  which  is  trying  to  fuse  segment 
matter  into  need-meeting  experiences.    The  break  between  content  and 
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method  is  both  awkward  and  miseducative  for  our  teachers.    The  type  of 
activities  you  have  used  seems  to  me  to  be  too  close  to  the  Evaluative 
Criteria  for  Secondary  Schools,  rather  than  an  example  for  the  revision 
of  this  work. 

I  do  believe  this  has  been  a  step  ahead  which  we  need.  However 
I  hope  you  will  basically  revise  your  criteria,  introducing  an  organ- 
ization and  a  structure  which  will  help  teachers  and  administrators  to 
move  in  the  direction  of  the  best  educational  knowledge  which  we  now 
have. 

It  seems  that  somewhere  there  should  be  a  place  for  evaluation  of 
activities ,  interpersonal  relations  between  staff,  staff  and  adminis- 
tration, staff  and  students,  staff  and  parents,  pervading  atmosphere  of 
school,  system,  and  classrooms,  schools  and  system.    Particularly  the 
latter  two  are  neglected." 

"Although  we  have  taken  a  somewhat  different  approach  than  many 
of  your  parts,  they  seem  satisfactory  for  general  use." 

"Since  the  personality  of  the  administrator  and  the  teacher  is  so 
important  in  the  teaching-learning  situation,  it  might  be  well  to  have 
a  check  list  on  personal  characteristics. 

Vlhile  it  is  necessary  to  check  on  each  subject  taught,  I  think 
some  provision  should  be  made  for  evaluating  large  units  of  work  which 
should  be  carried  on  to  give  children  experience  in  planning,  organizing, 
and  in  making  use  of  skills  taught  in  several  areas." 

"We  found  it  very  useful  and  informative." 

Speaking  of  Section  II,  "•  •  •  all  of  these  allow  for  much 
reasonable  variation.    The  criteria  is  provocative." 

"I  have  used  the  Elementary  Evaluative  Criteria  to  help  me  as  a 
principal— to  be  more  exact  in  my  goals— to  clarify  them  with  the  staff, 
superintendent  and  P,  T.  A.    It  is  a  definite  help. 

I  am  particularly  interested  in  seeing  that  we  meet  the  needs  of 
the  pupils  in  a  changing  community.    A  careful  check  has  been  needed; 
the  Evaluative  Criteria  has  answered  that  need." 

"I  think  there  is  some  helpful  material  in  your  program  and  I 
would  like  to  encourage  your  keeping  it  in  production  for  the  use  of 
public  schools." 

Pertaining  to  Section  II.    "I  find  these  areas  very  inclusive." 

"The  criteria  definitely  aided  in  improving  the  sequential 
relationships  of  the  subjects  mentioned,  mathematics  and  science.  It 
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gave  the  teachers  an  opportunity  to  cooperate  with  each  other  success- 
fully with  the  equipment  offered  for  teaching  purposes.    Thus  overlapping 
was  almost  completely  eliminated." 

"Section  II  in  your  questionnaire  was  answered  'satisfactorily' 
because  I  feel  your  brochure  indicated  a  work  carefully  considered  and 
well  developed.    However  in  any  reference  of  this  type  the  important 
aspect  is  the  encouragement  offered  to  a  staff  in  the  planning  of  a 
means  of  evaluation  that  derives  from  our  situation.    In  such  a  planning 
is  growth  and  understanding  and  that  is  when  our  basic  objectives  are 
served. " 


Checklists. 

"One  instruction  we  think  needs  more  emphasis  is  that  teachers 
may  talk  over  checklists  but  not  evaluation  questions  before  their 
evaluation  of  each  area." 

"Simplify  checklist  and  evaluations.    Marks  for  checklists  items 
could  be  letters  or  numbers  or  words — shorter  list." 


Arts  and  Crafts. 

"We  feel  that  the  arts  and  crafts  are  more  suitable  for  a  junior 
high  school.    This  section  has  been  revised  by  our  art  department  for 
our  use." 


Integration  and  Evaluation. 

"In  our  present  situation  in  many  elementary  schools  of  South 
Dakota,  I  feel  that  much  can  be  accomplished  by  taking  a  close  look  at 
the  various  areas  of  the  curriculum.    The  amount  of  training  of  the 
teachers  involved  does  not  warrant  too  much  attention  to  integration  and 
coordination  of  subject  areas  as  yet.    Perhaps  another  set  of  criteria 
for  use  by  schools  ready  for  the  next  step  might  be  in  order  rather  than 
changing  your  present  publication." 


Language  Arts. 

"Checklists  are  too  complete  and  detailed.    No  school  would  want 
to  be  able  to  check  affirmatively  all  items,  yet  the  impression  given  is 
that  the  more  items  that  can  be  checked  the  better  is  the  school. 

Many  items  are  applicable  only  to  certain  age  levels.    It  would 
be  advisable  to  group  primary  and  intermediate  grade  items  separately. 

Page  98,  omit  Community  Relations. 
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Page  101,  seems  unnecessary." 


Philosophy. 

"At  the  elementary  level  we  are  concerned  about  the  attention  we 
should  give  the  'core  program'  and  the  general  integration  of  related 
subject  fields.    Your  evaluative  criteria  in  neither  philosophy  section 
or  curriculum  section  makes  evaluation  of  this  type  of  program  possible 
or  perhaps  we  should  say  reasonable." 

Hie  following  comments  were  made  by  those  respondents  who  pur- 
chased five  or  more  copies  of  the  criteria: 


General  Mature  Comments. 

"We  have  found  the  material  a  little  'wordy. '    We  have  also  not 
been  able  to  follow  the  suggested  time  schedule,  as  discussion  of  so 
many  points  has  been  quite  lengthy.    It  is  an  excellent  publication  and 
we  are  enjoying  its  use  according  to  our  needs." 

"Seems  to  be  a  fine,  practical  attempt  to  give  concrete  aid 
rather  than  philosophical  malapropisms. " 

"I  want  to  say  that  your  instrument  is  easily  the  best  and  most 
complete  criteria  we  could  find.    Some  parts  of  it  were  supplemented  but, 
on  the  whole,  I  would  have  to  check  all  parts  of  it  as  being  satisfac- 
tory.   It  certainly  served  our  purpose  admirably.  ...  You  are  doing 
pioneer  work  in  this  field.    I  am  sure  you  will  make  changes  and  improve- 
ments as  you  go  on.    There  isn't  much  in  the  way  of  criteria  by  which  to 
measure  your  materials,  so  it  seems  essential  that  progress  or  change  be 
conservative.  ...  your  instrument  is,  in  my  opinion,  the  finest  work 
of  this  nature  available." 


Integration. 

"I  would  like  to  see  more  integration  of  art,  music,  and  physical 
education  with  the  other  subjects  possibly  through  the  unit  method." 


Philosophy. 

"Philosophy  and  objectives  could  be  developed  more  completely." 
"Too  little  emphasis  is  put  on  philosophy. " 
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Library  Services. 


"life  do  not  find  central  library  as  functional  as  the  rotation  use 
of  classroom  materials." 

"I  would  like  to  suggest  some  revision  on  the  Library  Service  in 
that  it  applies  mainly  to  a  centralized  library.    We  feel  that  more 
provision  could  be  made  for  evaluation  of  libraries  which  exist  in  a 
self-contained  classroom. " 


Checklists. 

"The  instructions  for  rating  checklists  and  evaluation  need  some 
revision.    We  found  that  some  teachers  were  too  critical  and  others  too 
lenient,  but  that  probably  can't  be  helped  by  you.    The  marking  'N1 
causes  confusion.    If  the  item  is  missing  it  is  often  passed  off  as  not 
needed. 

It  would  be  better  if  some  method  of  evaluating  the  degree  of 
integration  and  coordination  can  be  found. " 

"A  suggestion  we  would  like  to  offer  would  be  a  summation  sheet 
to  be  used  with  the  statistical  summary  for  each  section.    This  summary 
sheet  would  have  spaces  for  comments  listed  under  each  division  of  the 
statistical  summary.    We  found  that  definite  patterns  of  strengths  or 
weaknesses  were  evident  under  each  division  which  teachers  were  unable 
to  interpolate  statistically.    Each  staff  member  could  then  have  the 
results  fastened  in  the  manual  for  future  reference." 


Social  Studies. 

"Recommend  greater  emphasis  on  political  boundaries— which  in- 
fluence people  about  as  much  as  physical  environment— and  the  acquisition 
of  more  definite  facts:  names  of  countries,  their  capitols,  principal 
cities,  specific  industries  in  specific  places.    I  think  there  tends  to 
be  too  much  generalizing  and  skimming  in  social  studies,  with  pure 
geography  suffering  severe  neglect." 


Health  and  Physical  Education. 

One  respondent  feels  that  health  and  physical  education  need 
revision.    "Due  to  our  course  of  study  and  the  emphasis  on  outside 
activities." 
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CHAPTER  V 
SUMMARY  AND  CONCLUSIONS 


The  purposes  of  this  investigation  were: 
1«    To  determine  the  purposes  for  which  the  "Elementary  Evaluative 
Criteria"  was  being  used, 

2.  To  determine  how  the  criteria  has  aided  its  users,  in  what 
ways  and  to  what  extent. 

3.  To  determine  the  effectiveness  of  the  criteria  as  an  evalu- 
ating instrument  in  its  entirety  and  in  its  separate  sections. 

A  questionnaire  was  prepared  and  sent  to  1+93  purchasers  of  the 
criteria.    Of  these  U93  purchasers  168  of  them  bought  five  or  more 
copies,  136  purchasers  of  one  to  four  copies  replied,  and  seventy -nine 
purchasers  of  five  or  more  copies  replied.    The  data  obtained  from  the 
responses  to  this  questionnaire  were  analyzed  in  four  ways: 

1.  On  the  basis  of  all  replies. 

2.  On  the  basis  of  those  persons  purchasing  five  or  more  copies 
of  the  criteria. 

3.  General  comments  from  persons  purchasing  one  to  four  copies 
of  the  criteria. 

U.    General  comments  from  the  persons  purchasing  five  or  more 
copies  of  the  criteria. 

As  a  result  of  the  analysis  of  the  data  mentioned  above,  the 
following  conclusions  were  reached: 
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Conclusions  based  on  data  pertaining  to  all  replies. 

The  criteria  were  used  for  many  purposes.    The  main  purposes 

a.  Evaluation  of  a  single  school. 

b.  As  a  basis  for  discussion  at  teacher's  meetings. 

c.  For  use  in  study  groups. 

d.  As  reference  materials,  and  as  resource  materials. 
All  sections  of  the  criteria  were  considered  satisfactory. 

Negligible  responses  were  received  in  regard  to  having  any  area  of  the 
criteria  revised. 


5.    The  instructions  for  rating  checklists  and  evaluations  were 
overwhelmingly  considered  adequate. 

4.    The  statistics  indicated  that  the  subject  areas  need  further 


integration  and  coordination.    The  number  of  replies  that  indicate  this 

is  not  statistically  sufficient  to  warrant  this  as  being  conclusive. 

(For  comparative  purposes  see  conclusion  number  four  in  the  section 

below. ) 

5»    The  criteria  were  not  being  used  with  laymen  to  a  very  great 

• 

extent. 

6.    The  criteria  afforded  some  degree  of  improvement  to  the 

majority  of  its  users. 

Conclusions  based  on  data  pertaining  to  replies  of  purchasers  of 
five  or  more  copies. 

1.    The  criteria  were  used  specifically  to  evaluate  a  total  school 

system.    In  addition  to  this  main  purpose  some  of  the  respondents  also 

1. 


were : 


2. 


• 


i 


39 

used  the  criteria  for  reference  materials  and  other  miscellaneous  uses. 

2.    All  sections  of  The  Elementary  Evaluative  Criteria  were 

m 

considered  satisfactory.    There  was  no  indication  of  a  need  for  revision 

or  deletion  of  any  part  of  the  criteria  as  it  now  exists. 

5»    The  instructions  for  rating  checklists  and  evaluations  were 

considered  adequate. 

4.    The  majority  of  the  respondents  felt  that: 

a.  The  subject  areas  do  not  need  further  integration  and 

coordination. 

b.  The  subject  areas  are  suitable  as  set  forth  in  The 

Elementary  Evaluative  Criteria  for  the  majority  of  its  users 

• 

It  is  believed  that  this  conclusion  is  more  valid  than  number 

four,  in  the  section  above,  due  to  the  fact  that  the  replies  from  the 

respondents  who  purchased  five  or  more  copies  of  the  criteria  were 

giving  the  aggregate  opinion  of  a  total  school  system,  rather  than  the 

opinion  of  a  single  school. 

5«    The  Elementary  Evaluative  Criteria  is  not  being  used  with 

participating  laymen  to  the  extent  that  it  is  felt  that  it  should  be  used. 

6.    The  Elementary  Evaluative  Criteria  has  aided  its  users  in 

many  ways: 

a.  It  has  served  as  a  stimulant  to  the  creative  imagination 

of  its  users. 

b.  It  has  provided  worthy  objectives  for  schools  that  differ 

w 

in  their  philosophies  of  education. 

7»    The  comments  of  the  purchasers  indicate  that  the  criteria 

are  the  most  practical  of  its  kind. 
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8.    Hie  criteria  afforded  some  improvement  to  the  majority  of 
its  users.    The  number  of  respondents  who  indicated  that  the  criteria 
had  afforded  no  improvement  were  infinitesimal. 
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APPENDIX 

A.    QUESTIONS  PERTAINING  TO  USES  OF  CRITERIA 


I.  Do  you  use  the  Criteria  in  any  of  the  following  ways? 

A.  To  evaluate  elementary  education  in  a  total 
school  system  

B.  To  evaluate  a  single  elementary  school  .... 

C»  To  evaluate  a  phase  of  elementary  education  (e.g 
arithmetic,  language  arts,  guidance  services). 

D.  For  discussion  in  teachers1  meetings   

E»  For  use  in  study  groups,  as  resource,  or 
reference  material  •  

F.  Other    


PLAN 

TO  USE 
YES    NO  LATER 
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B.    QUESTIONS  PERTAINING  TO  CONTENTS  OF  CRITERIA 


II.  Listed  below  are  the  titles  of  the  various  sections  of  the  Criteria. 
Check  in  the  parentheses  provided  your  reaction  to  each  of  these 
sections. 

NEEDS 

SATISFACTORI    kevISION*  delete 

Manual  

Philosophy  and  Objectives   

The  Curriculum  • 

Kindergarten   

Arithmetic   

Arts  and  Crafts  

Health  and  Physical  Education  

Language  Arts  

Music.    • 

Science  

Social  Studies   

Library  Services  

Guidance  Services   

School  Plant  • 

School  Staff  and  Administration   

Individual  Staff  Member  Data  Sheet  
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APPENDIX 

C.    QUESTIONS  PERTAINING  TO  GENERAL  REACTIONS  TO  CRITERIA 


III.  General  Reactions 

YES  NO 

A.  Do  you  consider  the  instructions  for  rating  checklist 

and  evaluation  items  adequate?   ()  ()* 

B.  Should  increased  attention  be  given  to  the  integration 

and  coordination  of  subject  areas?   ()*  () 

C.  In  your  position  of  (  )  administrator,  (  )  supervisor, 
or  (  )  teacher,  have  the  Criteria  been  of  assistance 

to  you?   (  )      (  ) 

D.  Have  the  Criteria  been  used  with  lay  personnel?  ...      (  )      (  ) 

Degree  of  Improvement 
Extensive  Some  None 

E.  Has  use  of  the  Criteria  brought  about 

improvements  in  your  school  system?  ....         ()      ()  () 

Respondent  Investigator 

Name  Position  

Research  Assistant 

Addre  s  s  

*  Suggestions  for  revision  or  further  information  should  be  written  on 
the  reverse  side. 
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D.    LETTER  OF  TRANSMITTAL 


ELEMENTARY  EVALUATIVE  CRITERIA 
332  Bay  State  Road 
Boston,  Massachusetts 


Our  files  indicate  that  you  have  purchased  during  the  past  year 
copies  of  Elementary  Evaluative  Criteria. 

We  are  interested  in  finding  out  how  these  materials  have  been 
used,  how  effective  they  have  been,  and  any  suggestions  you  may 
have  for  revision  and  improvement. 

On  the  attached  sheet  is  a  series  of  questions  to  which  we  would 
like  your  candid  reaction.    Your  reactions  and  critical  evalua- 
tion of  this  publication  are  of  interest  to  us  who  have  developed 
it  and  will  be  of  definite  value  to  those  who  may  be  using  it  in 
the  future. 

A  self -addressed  envelope  has  been  provided  to  facilitate  your 
return  of  the  questionnaire  to  us.  We  would  appreciate  an  im- 
mediate reply. 

Thank  you  for  your  kind  cooperation  in  this  research  effort. 

Sincerely  yours, 


Research  Assistant 
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APPENDIX 
E.    QUESTIONNAIRE  FORM 


ELEMENTARY  EVALUATIVE  CRITERIA  INQUIRY 
I,  Do  you  use  the  Criteria  in  any  of  the  following  ways? 


PLAN 
TO  USE 
YES    NO  LATER 


A.  To  evaluate  elementary  education  in  a  total 
school  system  


B.  To  evaluate  a  single  elementary  school 


C.  To  evaluate  a  phase  of  elementary  education  (e.g. 
arithmetic j  language  arts,  guidance  services)*  • 


D.  For  discussion  in  teachers'  meetings 


E.  For  use  in  study  groups,  as  resource,  or 
reference  material   


F.  Other 


II.  Listed  below  are  the  titles  of  the  various  sections  of  the  Criteria. 
Check  in  the  parentheses  provided  your  reaction  to  each  of  these 
sections. 

NEEDS 

SATISFACTORY    revision*  delete 

Manual  

Philosophy  and  Objectives   

The  Curriculum  

Kindergarten   

Arithmetic   

Arts  and  Crafts  

Health  and  Physical  Education  • 

Language  Arts  

Music  

Science  

Social  Studies   

Library  Services  

Guidance  Services   

School  Plant  

School  Staff  and  Administration    • 

Individual  Staff  Member  Data  Sheet  ()  (  )  () 
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Ill,  General  Reactions 

YES  NO 


A»  Do  you  consider  the  instructions  for  rating  checklist 

and  evaluation  items  adequate?   ()  ()* 

B.  Should  increased  attention  be  given  to  the  integration 

and  coordination  of  subject  areas?   ()*  () 

C.  In  your  position  of  (  )  administrator,  (  )  supervisor, 
or  (  )  teacher,  have  the  Criteria  been  of  assistance 

to  you?   (  )      (  ) 

D.  Have  the  Criteria  been  used  with  lay  personnel?  ...      (  )      (  ) 


Degree  of  Improvement 
Extensive  Some  None 

E.  Has  use  of  the  Criteria  brought  about 

improvements  in  your  school  system?  ....         ()  ()() 

Respondent  Investigator 

Name  Position  

Research  Assistant 

Address         


*  Suggestions  for  revision  or  further  information  should  be  written  on 
the  reverse  side. 
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